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Properties of Sums of Pseudo-Random 
Variables in Feedback Shift Registers 


0.0 Introduction 

It is well-kno\-m that the output of a binary feedback shift register, 
such as that shown in Figure 1, is a binary pseudo -random sequence. 



Figure 1. Binary Feedback Shift Register 

The statistics of the output sequence have been studied in some 
detail, Other researchers have studied the important delay or 
shifting property^^^^"^^^^ , the statistics of the output after filtering 
the spectral density of the output some practical applica- 
tions , and finally a number of other closely related ideas 

This report is concerned with the statistics of the sum of the values 
in the shift register ~ . The problem is considered here from a 
number of viewpoints. We first consider in section 1 partial results 
obtained from a computer study. These results show a number of interesting 
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features. To some extent, sections 2 and 3 are intended to explain some 
of the features observed in section 1. The major conclusion from the 
computer study is that the generated distributions become extremely complex 
as the number of stages in the shift register and the degree of inter- 
dependence increase. 

In section 2 we consider the problem of the sums of weighted outputs 
of the stages of the feedback shift register. A method of synthesizing 
pseudo-random distributions is developed. It is shown that several 
inportant distributions such as uniform, triangular, stairstep and bimodal, 
as well as an unlimited number of other less important distributions, are 
easily synthesized. 

Section 3 is concerned with the sums of the outputs of the feedback 
shift register and various delayed or shifted outputs. The results of • 
sections 2 and 3 are shown to be closely related. 

In section 4 we study the problem of the infinite sum of weighted 
random variables. Some important relations with the problems of the first 
three sections are shown. 

1. Computer- Generated Frequency Histograms 

We are concerned in this section with the frequency histogram of the 

sum of values of outputs in the first M stages of a feedback shift 

register such as that sho^m in figure 1. The range on M is 0 to 

2^ - 1 . Where M exceeds n , it is assumed that additional delay stages 

are added to the shift register outside the feedback loop. The frequencies 

% 

of occurrence of different sums, over different values of M , were determined 
for n ranging from 3 to 11. The results are easily displayed in the form 



o£ a "frequency histogram matrix," as seen for n^ = 3 in table 1 below. 

S/M 0 1 23.4 5 6 7 

0 
1 
2 

3 

4 

Table 1. Frequency Histogram Matrix - n = 3 

The frequency of sums S appears as a row under each value of M . 

For example, consider the case where we add two successive stage outputs 

3 

at a time 0^1 = 2) . In a single complete period (length -2 -1 = 7) the 

sum will be zero once, one four times and two twice. 

Note that the matrix is symmetrical in that the right half repeats 
(see Table 1), in reverse order, the left half. To see why this is so, 
consider two columns of symmetry designated by M and 2^ - 1 - M . 

Note that these columns cover a total number of stages equal to 2^^ ^ 

(their sum) . Also, note that when we cover all 2^-1 stages the number 
of I's is 2^'^ . Hence, obtaining a sum S in column M is equivalent 
to obtaining a sum 2^’^ - S in column 2^ - 1 - M . Thus, the matrix 

must be symmetrical and it is necessary only to display the left half. 

We will' also drop the M = 0 term since it is not of general interest. 

With these changes the matrices for n = 4 and n = 5 are given in 
tables 2 and 3. 

It is of interest to briefly consider the matrix in table 3 foi n = 5 
as an example of a relatively complicated and yet manageable matrix. The 




S/M 1 2 3 5 6 7 


0 

1 


7 3 1 

8 8 6 


2 


0 1 + 6 


3 

h 


0 

0 


0 2 
0 0 


5 0 0 0 

6 0 0 0 


0 0 0 0 

2 1 0 

6 k 2 2 

h 6 6 k 

13 5 5 

0 0 1 1 | 

0 0 0 0 


7 0 0 0-0 

8 0 0 0 0 


0 0 0 
0 0 0 


Table 2. Frequency Histogram Matrix - n ~ h 



S/M 1 2 3 k 5 6 7 8 9 10 11 12 13 3 15. 

0 15 731000000000*00 

1 l 6 16 12 853100000000 

2 08 12 12 10 66520000 0 0 

3 00 )+ 8 l 0 10 61t5520000 

'4 000259975566200 

5 0000139 12 10 6 4 4651 

6 00000003 8 •'084 4 ST 

,7 000000 . 0 ul 5 lu 06 

8 OOOOOOOO 0-015995 

9 OOOOOOOOOOOi. - 59 

10 00000000000“' 13 

Table 3. Frequency Histogram Matrix - n = 5 
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first n colunuis follow, except for one zero term the well-taiovm and 
easily expressed binomial frequency law. (See section 2) A fairly 
concise expression is also available for the n + 1 column. For 
greater values of M no easily handled explicit expression is Imo^m. 

The problem is that if M > n the summed terms become dependent and 
their frequency histograms highly complex. One of the objectives of 
sections 2 and 3 is to explain how this dependency arises and what kinds 
of distributions or frequency histograms arise in this and other kinds 
of summing techniques. 

We do not have space to present frequency histograms for larger 
values of n . It is of value, however, to observe that as n increases 
and M increases the histograms become increasingly complex and apparently 
without order. This reflects the high degree of interdependency of terms. 
Consider, as an example, the histogram H(sj for n = 8 and M = 67 . 

i 

s 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 

H(s) 3 16 27 28 8 11 17 26 33 26 19 18 ‘ 20 2 1 

This is the 67^^ column of the n = 8 matrix. It is clearly very 
complex. There is no apparent order or structure in this matrix. 


2.0 Weighted Sums from Feedback Shift Registers 

In this section we consider a weighted sum of outputs from a feedback 
shift register. The basic system is shorn in figure 2. It v/ill be assumed 
that the feedback connections are such that maximal -length binary sequences 
of length 2^-1 are generated. 



n 


i 




Figure 2. Weighted Sums 

The required connections will not be considered here. The n stages 
o£ the feedback shift register can have values 0 or 1 (arbitrary units) 
during any specified clocking period. As the shift register cycles through 
a complete period, each stage has a sequence of O’s and l*s follomng a 

m 

well-known '■ ■' and definite pseudo-statistical pattern. 

* r 

The term "pseudo” is used here and in other parts of tlie paper to 
indicate that the outputs or variables appear to be random and thus capable 
of a statistical description. Actually, we are dealing with a finite-state- 
machine vath a prescribed initial state and defining relationship, and the 
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output is thus knom deterministically. Nevertheless, it is of great 
value to treat the output as random in some sense. We will adopt the 
practice here of speaking of probability distributions even though a 
term such as * 'deterministic frequency histogram" might be more correct, 

2.1 The Binomial Distribution 

In the diagram of figure 2 the weighting factors, a^ , can take any 
integer value. Consider first the case where a^ = 1 for all j from 
1 to n . In this case, we are simply adding the number of outputs of 
value 1. It will be shorn in section 3, when we discuss the generating 
polynomial, that the outputs of the stages are independent of each other. 
Since the stages can have two possible values, the sum of n stages must 
have a binomial probability distribution. Actually, there is one state, 
the all- zero state, which cannot occur. The result is that the distribution 
is binomial except for one term. With the probability of 0 or 1 equal 
to a half, the distribution of the sum becomes: 

PCs) = C^)~ for 1 < s < n (1) 

s zH . 1 - - 

where stands for the combinations of n things taken k at a time. 

In much of the discussion to follow the all- zero state will be assumed 

possible. This simplifies calculations significantly. It will be removed 

when final results are presented. iVhen the all-zero state is assumed the 

distribution is the familiar binomial. A series of binomial distributions 

for increasing values of n is easily presented by means of the Pascal 

f391 

triangle as shown in figure Z. ■' 
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n 


15 


10 10 


20 


15 


21 35 35 21 


28 56 70 56 


28 


n 

n 

n 

n 

n 

n 

n 

1 n 


Figure 3. The Pascal Triangle 


Once again n is the number of stages. Thus, for n = 5 stages we 
can use the n = 5 row to find the binomial distribution of the sum of out- 
puts in the five stages (including the all- zero state) . To use the Pascal 
triangle for n =* 5 we first normalize by dividing each term in the n “ 5 
row by the sum of the terms, which is 32(2^), The first term on the far left 
is the sum equals zero term. Thus, the probability that the sum equals zero 
is 1/32. The probability that the sm equals one is 5/32, etc. The complete 
distribution is : 


S 

P(s) 


0 1 i 2 3 

1 10 10 

TZ 32 32 32 


4 5 

_5 J. 
32 32 


2 . 2 Sums of random variables - Discrete Convolution 


IVe ^d-sh now to consider the very interesting case where the a's are 
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not all one. In the work which is to follow we make use of a mathematical 
approach and a nomenclature which are not too common. Vfe will, therefore, 
review these ideas first. 

We will write the probability distribution as a sequence of unnormalized 
numbers. For exaiiple, the binomial distribution for n = 5 is: 

[1 5 10 10 ' 5 1 ] 

This is intended to convey exactly the same information as the 
distribution at the end of section 2.1. 

Recall now that we are basically interested in obtaining the analog sum 
of some independent random variables (in the stages of the feedback shift 
register.) It is well-knom (see for example [40] , Chapter 15) that the 
probability distribution of the sum of two random variables is the convolution 
of the two original probability distributions. It is particularly easy to 
convolve two discrete probability distributions using the sequence format 
shown above, (see [41J or chapter 3 of [40]). As an example let us find 
the distrxButibn of the slmT of "binomial random variables for n = 3 and 
n = 4 . From the Pascal triangle we see that the correct sequences are: 

[1 3 3 1 3 f or n = 3 

[1 4 6 4 1] forn=4 

There are various ways to mechanically carry out this convolution. 

Perhaps the simplest is to set up a format much like that of conventional 
multiplication. The only difference is that with this approach we do not 
carry tens. 
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1 4 6 4 1 

13 3 1 

"I 4 6 ? r 

3 12 18 12 3 

3 12 18 12 3 

14 6 4 

1 7 21 35 35 21 7-1 

Using the symbol ^ for convolution we write the above problem in the form: 

[1 4 6 4 1]«[1 3 3 1] = [1 7 21 35 35 21 7 1] 

Ofe might note that the solution to this convolution problem is a sequence 
equivalent to the binomial distribution for n = 7 . The property of 
obtaining the distribution for order n by convolving any two distributions 
whose orders add to n holds in general for the binomial family, as well 
as for a number of other families. It does not hold for all families of 
distributions . ) 

2,3 The Uniform Distribution 

With the above technique we can now obtain the distribution of the sum 
of weighted outputs from stages of the feedback shift register. 

As a first exanple consider the simple case where n = 2 , = 1 and 

a 2 = 2 . The weighted output of the first stage is 0 or 1 with equal > 
probability of 1/2 Owe are assuming the all-zero state for the present) . This 
distribution is expressed as: 

[1 1 ] 
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The weighted output o£ the second stage is 0 or 2 with equal 
probability which w'e express as: 

[1 0 1 ] 

That is, zero appears with probability 1/2, one with probability zero 
and two with probability 1/2. All terms below zero and above the greatest 
term specified in the sequence occur with probability zero. To find the 
distribution for both stages, out of the analog adde:^ we convolve the above 
to obtain: 


[1 1 ] ^ [1 0 1]=[1 1 1 1 ] 


This says that the sums 0, 1, 2 and 3 occur with equal probability 
1/4. This result can be checked by means of the following table. Call the 
state of the stages and ^2 • 


k 


1 


0 


0 

1 

1 


0 0 

1 ' 0 
0 1 

1 1 


^ 2^2 

0 

2 

0 

2 


h^l ^2^2 
0 
2 
1 
3 


Thus we see that each term appears just once for the four distinct and 

equally probable combinations of states k^ and k2 . h’e can now make use 

of the fact that the all zero state is not possible and the maximal- length 

2 

sequence has a length 2 -1=3. Without the all zero state our 

distribution becomes 


[ 0 1 1 1 ] 
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Now let us add a third stage with weight factor 4. First we go back 
and restore the all zero state until the problem is completed. The - 
distribution of the weighted third stage is: 

[ 1 0 0 0 1 ] 

That is, the values 0 and 4 occur with equal probability 1/2, 

and the values 1 , 2 and 3 occur with probability zero. To find the 
distribution of the sum for three stages \ie convolve the distribution of 
the third stage with that of the first two. 

[1 1 1 Ill'll 0 0 0 1] = [1 1 1 1 1 1 1 13 
and removing the all zero term we have 

f 

[ 0 1 1 1 ;, 1 1 1 1 ] 

which says that the integers one through seven occur with equal probability. 

If we add fourth stage with weight 8 the distribution suggests that the 
first 15 integers are equally likely to occur. Appropriate w^eightings 
cause this pattern to be continued. Thus we come to the most interesting 
conclusion that a uniform pseudo-random distribution can be generated by a 
feedback shift register with suitable weights. It is apparent that the 
proper wciglit for the 3 ^ stage is 2 ^” if a uniform distribution is 
desired. 

2.4 Some Additional Non-Binomial Distributions 

The possible variations suggested by this basic approach are endless. 
We mention here only some of the most important. 

Consider a six stage register with weights a^ = a 2 = 1 , = 2 , 
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= 4 . We have just seen that the combination of weights 1> 2, 4 
produces a distribution, 

[ 11111111 ] 

i 

But now we have two such distributions so we convolve them together to 
obtain: 

[ 11111111 ]*[ 11 - 111111 ] 

= [123456787654321] 

The result is a triangular (Simpson) distribution. (The pertinent weights 
and final distributions are summarized for this case and other important cases 
in table 4.) 

A "stairstep" distribution is obtained from, a combination of v/eights such 
as 1, 2, 4, 4 which give. 

[ 111111 ' 11]*[1 0 0 0 1 ] 

= [ 111122221111 ] 

Finally, a "Bimodal" distribution is obtained from weights such as 

I 

1,1, 1,1, 5 which givel? 

[14641]^[100001] 

= '[ 146411464'l] 

The peak of the second mode can be extended out to any value by simply 
increasing the last weight. This bimodal technique can be used on any type 
of distribution by simply adding a stage with a weight equal to the distance 
the second mode is to be shifted. 


Name 


Weip;hts 


Distribution 


Histogram 



^2 





1 


Binomial 1 1 1 l[o461il] 



Uniform 1 2 it I t { i I I I i I I f * * 


Triangular 1 2 1 2[0 23it32l] 



Stairstep 1 


[0 


1 .‘ 


1] 


1 I i 


U 


I I I I 


Bimodal 1 1 1 5 [0 33-10133 1] 



Table 4. Synthesis of Distributions of Pseudo-Rdndom Variables 


Cn 
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The preceding examples are summarized in table 4. Necessary weights 

and the resulting distributions are sho\m. The histogram is simply a 

pictorial representation o£ the distribution. 

Once again it should be emphasized that the possible variations to the 

\ 

above are unlimited. Any number of different distributions can be_ generated 

simply by changing the weights, a. . 

' 1 

Although the above technique for synthesizing pseudo-distributions is 

very attractive it is not without limitation. Consider the uniform dis- 

tribution, generated by weights 2^~ . The range of this pseudo-random 

variable is from 1 to - 1 where n is again the number of stages. 

Thus, for example, this simple circuit cannot be connected to yield a 

pseudo-random variable uniform between 0 and 9 . The latter \^ould be 

necessary to develop a decimal range pseudo-random number generator. 

r42l [431 

These results have been reported on in the literature '■ ^ . 

2.5 Some Thoughts on Range and Smoothness 

In this section we are interested first in the range, of the pseudo- 
random variable S , the output from the circuit of figure 2, and second 
on the effect which w'eighting has on "smoothness" of the output distribution. 
These factors are of course inter-related. Note that ve define the range 
here as the number of possible outcomes or values of S , rather than the 
difference between S nrax and S min . 

The range of S can be in^ortant for a number of applications as it - 

[441 

indicates the number of different output levels of the device. Aggar^^'al^ , 
for example, has suggested a frequency generator which generates a number 
of frequencies equal to the number of different levels of S . 

If we consider the circuit of figure 2 vdth unity v;eights, the range 
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* 

on the sum is n since the sum can be any number from 1 to n , with a 
distribution following (almost) the binomial law. If we let the weights 
be 2^ ^ , we obtain a unifom distribution with range 2^ - 1 , as we 
saw in section 2.3. This latter range is obviously the maximum range for 
the circuit of figure 2, since after 2^-1 outputs the periodic sequence 
repeats. Likewise n is the minimum range. Hence, the range r^ has 
bounds : 

n ^ Tg 2^ “ 1 

where the weighting sequence for the lower limit is [1,1, 1, •••!], 
and for the upper limit [1, 2, 4, 8, 2^^’^ ] . For any weighting sequ< 

[ , a 2 , • • • > 1 "tbe range r^ is simply the number of different numl 

which can be obtained by adding any combinations of the weights tahen anywl 
from 1 to n at a time. 

The analysis in sections 2.3 and 2.4 raises some interesting question; 
related to the Central Limit Theorem tendencies of sums of random variable; 

- -The Central Limit Theorem says that the distribution of the sum of continu< 
random variables tends to be Gaussian (or normal) as the number of summed 
variables increases, under certain conditions. A discrete approximation 
to the Gaussian is the Binomial distribution. The accuracy of the approxit 
increases as the order of the binomial increases. The Gaussian distributi( 
is in a sense the ''smoothest" possible continuous distribution. In the sai 
general sense the Binomial distribution is the smoothest discrete distribu- 
Summed discrete random variables tend to be Binomial under conditions 
analogous to those necessary for the Central Limit Theorem. 

Consider what happens when use weights of unity in all stages of tl 
circuit of figure 2, The distribution of each stage is [11] and the 
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distribution of the sum for an n-stage circuit is binomial of order n . 

Consider now 't\/hat happens as we start to use non-unity weights . • Let 
n = 3 and find the distribution for a series of weight combinations as 
shown in table 5. Assume the all- zero state. 


a^ a^ 
2 o 


■2 


1 

1 

1 

1 

1 


1 

2 

2 

2 

2 


1 

1 

2 

3 

4 


1 

1 

1 

1 

1 


3 

2 

1 

1 

1 


3 

2 

2 

1 

1 


1 

2 

2 

2 

1 


0 0 0 0 

l' 0 0 0 


1 

1 

1 


1 

1 

1 


0 0 


1 

1 


0 

1 


Table S. Effect of Weights on "Smoothness” 


As the weights increase the range increases and the "smoothness" 

decreases. The ordinary (normal?) tendency under summing is for the 

distribution to become increasingly smooth. However, the tendency of the 

weights is to emphasize individual distributions. These two tendencies 

0 12 

tend to 'offset each other. If the weights are chosen as 2 , 2 , 2 , 

•z 

2 •• as in the last line of table 5, the two tendencies cancel each 

other and we retain the uniform distribution. 

It is of interest to consider the sum which leads to the uniform 
distribution in more detail. First, we recall that the condition for the 
Central Limit Theorem, the Lindeberg-Feller condition, can be expressed in 

4 

simple form as. 


lim 

n->« 



n 

u. 

j=l ^ 


( 2 ) 
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2 

where is the variance o£ the j random variable. 

This condition is essentially applicable to the discrete case as well 

to the continuous case to the extent that the binomial distribution 

approximates the Gaussian. Let us consider the limit in ( 2 ) for the 

0 12 3 

case where the weights are 2 , 2 , 2 , 2 , • * * • . The corresponding 


distributions are: [1 1], [101], [10001], 


and the corresponding 


variances are 1/4, 1, 4, 16, 


4^' ^ where j = 1, 2, 3, ••• . 


The ratio of the variance of the j variable to the sum of the 
variances of the first n variables is: 


n . ^ 

14^-2 

j=l 


= 5x4^ 


The largest variance appears for j = n in which case the ratio is 


R. = 




And the limit is; 


lim = 0.75 
n^ 


Thus, for this case, the Lindeberg-Feller conditions are not met. This 
is, of course, exactly as we would have expected. A similar analysis for 
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senes o£ weights 1, 2, 3, 4, .... indicates the limit is zero. Thus, 
these weights are not large enough to offset the central tendency of the 
Slimmed random variables. Vfe will return to this question briefly when we 
discuss infinite sums in section 4. 

2.6 The Autocorrelation Function 

In the preceding work, it was convenient to let the states take 
1 

values 0 and 1 . c,<In our study of the autocorrelation function, we 
will let these values be -1 and +1 . Vfe are interested in the stream 
of values out of the analog adder in figure 2. This can be thought of as 
a stochastic process X\'ith a new value occurring at each clocking time. We 
will designate this sequence of output values as: 

[ b2, ••• , bj^, , b2n_j^ ] 

We will only consider 2^-1 terns since the sequence is periodic and 
repeats after 2^-1 terms. We define the autocorrelation function as: 

2^-1 

RCm) = I ^k ’ m=0,l,2,*-- (6) 

k=l 

(Note that because the sequence is periodic b^. 0 

This is also equal to the number of agreements (same sign) minus the number 

I 

of disagreements when the original sequence is compared with itself shifted 
by m units . 

We consider first the case where all the a*rs except one in figure 2 
equal zero. The one exception is equal to 1 . For this case, it is 
known tliat the autocorrelation function is: 



R^(m) = 


21 


2^-1 m=0 (7) 

-1 m ?^0 

This function which also has period 2^-1 , is plotted in figure 4. 
Note that we are considering m as a discrete integer variable. The 
graph represents an. integer as all of the space bet\^een the integer and 
the next higher integer. 



Figure 4: Autocorrelation Function of Maximal -Length 
Pseudo-Random Sequence 


From figure 4 we see that as the length or period of the sequence 
(2^-1) increases the autocorrelation becomes increasingly concentrated at 
m = 0 . 

Now we wish to study this autocorrelation function for the stream of 
values out of the analog adder in figure 2 for some non-trival weightings. 
As an example let = 1, E 2 = 1 and all other weights be zero. Then 
the sequence out of the generator is: 


[ + bz, b2 + bj, \ **’ > ^ 2^-1 ^ 


with autocorrelation function: 
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2^-1 

^2 ^ ^^k+in ^ ^k-nn+l^ 


2^-1 

" k=l ^ ^k+lVM^ ^k-nr* 

= 2R^(m) + Rj^Cm-1) + R^(ni+1) (8) 

Thus we have established the very interesting and useful, though hardly 
surprising, result that the autocorrelation function of the sum of sequences 
is a sum of the individual sequence autocorrelation functions. This discrete 
result is analogous to the well-knom^^^^ expression for the autocorrelation 
of a sum of continuous random "variables : 

r 

h 

Consider now the sum of p sequences obtained by letting a^ = a 2 = 

... t= = 1 and all other a.’s are zero. The autocorrelation function is 
P 1 

RpCm) = pRjCm) + Cp-l)R^(m-l) 

+ (p-l)RjCin-^l) + CP- 2) Rj^ Cm- 2) + ••• , 

+ R^Cii^-p+1) + R^C^‘*‘P+1) CIO) 

* t 

This result is a fairly straightforward extension of the p = 1 and 
p = 2 cases. It simply indicates the number of cross -terms of given delay 
obtained in the multiplication: 

^k'^^k+l'^ * * ’ ‘‘‘^k+1 J +***•*■ \'-im+p-l^ 



Equation (10) reduces to the following simple relation for the auto- 
correlation of the sum of p successive unity-weighted pseudo-random 
sequences. 

R (m) = (P-lml)2^ - , m = 0 , + 1 , + 2 , • • • , + P 

P ^ . (11) 

-P^ , m = P , P + 1 , • • • , 2^^ - 1 - P 

4 

This function is plotted in figure 5 for n = 4 and p = 1,2, 3, 4, 5, 6, 7 
and 8 . The effect of summing p sequences is seen to be to introduce 
"memory” into the output process. The process "remembers" over a number of 
clocking pulses equal to P , the number of successive sequences added. 

The effect is analogous to passing the original sequence through a low-pass 
filter. In fact the system is actually a non-recursive digital filter. 

It is, of course, not necessary to add successive stages or to use 
unit (or equal) weights. We consider first the sum of non -successive 
sequences and second the sum of sequences with different weights. 

Let a^ = a^ = 1 and all other weights be zero. 

2^-1 

\+m+2-^ 

( 12 ) 

= 2Rj^(m) + R^(m-2) + R^(m+2) 

And, similarly: 

R^^(m) = 2R^(m) + R^(m-q) + R^^(m+q) (13) 

til 

is the autocorrelation function of the sum of a sequence and the q later 

th 

sequence. The effect of delaying the weighting to the q stage is to 
delay the "memory" in the process. As an example Rj^^(m) is plotted in 




Figure 5, Autocorrelation Function - 

(scale not consistent for different values of p) 
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figure 6. 



Figure 6 


1 

h 

ii 

Other interesting autocorrelation functions can be obtained by adding 
sequences mth different delays. 

We turn now to the effect of non-unit weights. Let a^^ = 1 and 
a2 = e . The sequence out of the generator is; 


[b^ + eb2> b2 + eb^, 


b, + eb 


k+1’ 


^ 2^-1 


+ eb 


k+m+1 


) 


and its autocorrelation function is: 


R2(m) 


2^-1 


a,=l 

a2=e 


” L. ®^k+l^ ^k+m ®^k+m+l^ 


k=l 


= (1+e jR^(m) + eRj^(m-l) + eR^(m+l) 


(143 


This function is plotted in figure 7 for e = 1,2,3, and 4 . The ' 
important difference between these results and those of figure 5 is that in 
figure 5 the autocorrelation decreases in equal steps (triangular) whereas 
in figure 7 the function drops off more rapidly for m near 0 than for 
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Figure 7. Autocorrelation Functions for Different Weights 
(Scale not consistent for different values of e,) 
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larger m . The latter function resembles an exponentially decreasing 
function. As e increases the autocorrelation function resembles 
increasingly the autocorrelation function of the original pseudo-random 
sequence. This is certainly reasonable because values of e much greater 
than 1 simply emphasize the heavily weighted sequence. The important 
point here is that the process of weighting does effect the autocorrelation 
function and does so in a way which is intuitively reasonable. 

Let us consider the effect on the autocorrelation function of binary 
weights =1, a^^Z, a 2 = 4, ’*0 • 'This function is plotted in - 

figure 8 for a number of combinations of binary weights. The functions 
fall off in an ejq)onential-like way with each succussive step being about • 
1/2 the previous step when all levels are referenced not to zero but to 
the minimum level. 
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Figure 8» Autocorrelation Functions for Binary Weights 
(Scale not consistent for different wei^ts.) 
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3.0 Augmented Feedback Shift Registers 

We turn now to an alternative approach to changing the distribution 
obtainable from a feedback shift register. Actually there are two ways 
of implementing this approach. It will be sho^vn that these ways are 
related to each other and likewise are related to a lesser extent to the 
approach just discussed in Section 2.2. Before we consider the two ways 
available under this alternative approach we must briefly consider the 
defining or characteristic polynomial for the feedback shift register. 

3.1 The Characteristic Polynomial 

The state of a feedback shift register is completely specified by its' 
initial conditions and a so-called characteristic polynomial which 
interrelates outputs from various stages and the input to the first stage. 
One form of the characteristic polynomial for a feedback shift register 
with n stages connected to generate a maximal length sequences is* 

d’^ 0A^_1 •••©Aj^D^0D° = 0 - CIS) 

where 0 stands for the modulo 2 sum. This operation is defined in the 
table below. 


0 

0 

1 

0 

0 

1 

1 

1 

O' 


Table 6 


The coefficients A^ are either 0 or 1 depending on the feedback 
connections in the shift register. D is an operator signifying delay of 
order equal to its exponent. To clarify this general expression consider 
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the particular circuit shown in figure 9, and its characteristic polynomial: 



Figure 9. Five Stage Binary Feedback Shift Register 

D^©D^©D° = 0 . , (16) 

« ^ 

Note first that there is no need or use for a minus sign in modulo 2 
arithmetic since -1 and +1 are equivalent. Therefore we can put any term 
in equation (16) on the right side of the equation without changing signs. 
Thus, for example, we can wrrite: D^0D^ = jf . (17) 

The interpretation of this expression is as follows. The value in a 
certain stage is equal to the modulo 2 sum of the value tvro time delay 
periods earlier and the value five time delay periods earlier. Since the 
shift register shifts values or outputs one position each delay or clocking 
period, this is equivalent to saying that the modulo 2 sum of the value 
in the 5th and 2nd stages is equal to the value in the 0^^ stage. The 
latter is a hypothetical stage which feeds the first stage at a clocking 
instant. 

Since D simply stands for delay v:e can increase the exponent of each 
term in (17) and still have a correct equation. Thus, for example 

D^©D^ = (18), 

f 

Similarly, this- can be repeated any number of times, a property which 
will be very useful in the sections to come. The characteristic equation 
also tells us that values in stages are independent as long as the stages are 




31 


no more than 5 delay periods apart. In general, values are independent 
if stages are no more than n delay periods apart where n is the number 
of stages in the feedback loop. 

3.2 Added Stages 

We are ready now to consider the first of t\\'o alternative i\'ays to 
change the distribution available from a feedback shift register. Consider 
the circuit of figure 10. 



output 


Figure 10. Augmented Feedback Shift Register 


The basic feedback shift register is identical to that of Figure 9. 

To this we have added m-5 delay stages which are part of the shift 
register but are not in the feedback loop. In much of our discussion to 
follow, we will assume that the weighting factors related to the stages 
in the feedback loop are all equal to unity. 

Consider as a first example the problem where m = 31 in Figure 10. 
Let a^ through a^ equal one and all other factors equal zero. That is, 
we are adding the outputs of the five stages plus one additional stage 













TO 


delayed one clocking period from stage S. Call this analogsum "S^”. 

1 ? 4. 

Sg = D-^ + D + + D + + D (19) 

We know from section 2 that the sum of the first five stages is 
binomial. But the addition of a sixth stage adds a dependency which makes 
the resulting sirni non-binomial. To see this we substitute from 
equation (18) into equation (19) and collect terms to obtain: 

Sg = + (D^©D^) = + 

[D^ + + (D^©D^)1 (20) 

The terms within the bracket were collected because they are inter- 
dependent. The total bracketed term and the first three terms are independent 

and the resulting distribution of the sum can be found by the convolution 

2 4 

techniques developed in section 2. The distribution of the terms D , D 
and is binomial of order 3. That is, it is of the form. [0 3 3 1]. 

The distribution of the complex term in the brackets' cannot be found 
from convolution techniques. It is possible, however, to find the 
distribution using a table of possible states as shorn below. 





0 

0 

0 -0 

0 

1 

1 2 

1 

0 

1 2 

1 

1' 

0 2 


•? 

1 

1 

J 

f. 

Table 7 

i 

The resulting distribution of [D^ + + (D^ + D^)] is: [103], 
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^^tien we convolve this with the distribution for and we 

finally obtain: 

[1331] « [103] = [1 3 6 10 9 3] 

(-see Column 6 of Table 3) . 

If we now change the problem so that the output of the seventh stage 
is added to the first five we have: 

Sy = = D^.+ + [D^ + + (D^©D^)] 

( 21 ) 

and the resulting distribution is as in the previous example since the 
structure is the same. 

Now, however, consider the case where the added term is in the nintli 
stage. Then wre have: 

Sg = + D® 

= + D^©D^ 

= + D^0D^0D^ 

= + [D^ + + (0^00^00"^)] (22) 

In this case the dependency is in three stages rather than tx\ro and the 
distribution changes. Using the basic approach above, \/ith an 8 column table 
in this case, we finally obtain the distribution, [1 2 7 12 7 2 1], or, 

dropping the all- zero term, [0 2 7 12 7 2 1] . 

After two examples we are led to the general question, xs’hat is the 
distribution for. + D^" where j can be any 
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nLimber from 1 to 31? First, we should establish our range of interest 
on j as 31 (or 2-1). Vfe will not exceed 31 since any value of j 
greater than 31 is equivalent to some j betoeen 1 and 31 . This 
follows from the fact that the maximal -length pseudo random sequence 
repeats every 2^-1 (2^-l==31 in this case) clocking periods. Furthermore, 
we note that every involves a different combination of the first 
five stage terms. For if the term consisted of exactly the same 
combination of (D^ , and D^) for different values of 3 the 

sequence would have a period equal to the difference of the values of . j in 
question. Thus there are 31 different combinations of terms arising in 


S 

J 


which should be considered in determining the possible distributions 


obtainable. 


Not all of these combinations produce different distributions, (ife saw 


above that and had the same distribution) . Since the distribution 
is dictated by the structure, which reflects the number of interrelated 
terras, it is fairly obvious that there are 5 possible distributions 


depending on whether there are 1, 2, 3, 4, or 5 interrelated. terms, in a. 


particular combination. 

The question of how many ways there are to obtain a particular distributior 
is easily answered. There is just one way in which all five terms can be 
interrelated. Thus, one of the 31 possible values of j leads to the 
distribution typical of the case where all five terms are interrelated. 

There 'are 5 distinct ways of combining four dependent terms. That is, 
the combinations of five things taken four at a time is five. And, in fact. 


the combinaterial law is used to find the number of values of j which lead 


to a particular distribution. The distributions which are obtained by 
adding one additional delayed value are summarized in the following table 8 . 



35 


Included for reference is the binomial case where there are no dependencies. 
In all cases the all- zero term has been omitted. 


Number of Interdependent 
Stages 

0 

1 

2 

3 

4 

5 


Number of Ways to 
Obtain Dxstribution 

1 

5 

10 

10 

5 

1 


Distribution 
[0 5 10 10 5 1] 

[0 4 7 8 4 1] 

[0 3 6 10 9 3] 

[0 2 7 12 7 2 1] 
[0 1 10 10 5 5] 

[0 0 15 0 15 0 1] 


Table 8 


The above example is limited of course to a shift register \d.th five stages 
in the feedback loop and to a single added stage augmenting the basic five 
stages. It is of interest to generalize and expand our development. 

Consider first the case where we allow more than one additional stage. 
Thus, for example, we might consider the sum: 

1 2 3 4 5 6 7 


= + [D^ + + (D^ + D^)] + [D^ + + D^)l (23) 


The bracketed terms, identical in structure to the bracketed 
equation (20), have the distribution [1 0 3] as shown earlier, 
total distribution of is: 


term in 
Thus, the 


[1 IJ « [1 0 3] * [1 0 3] = [1 1 6 6 9 9] 


or, dropping the all-zero term: 


[0 1 6 6 9 9] 


i 


(see column 7 of table 3) 
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Tlus is a new distribution as we might have expected. Once again, 

there are a large number o£ different ways of adding 2 more values. Thus, 

many different distributions can be obtained depending on which ti\'o delayed 

values are selected. We can, of course, extend the number of added stages 

to 31 with a very great many different distributions obtainable. 

A second way of expanding on the possible distributions is by letting 

the weighting factors, , take values other than 1. (In section 2, we 

essentially covered this situation for the case where j was between 1 and 

n 0 Allow'ing A. to take value leads to literally countless different 
^ f 

distributions . | 

A final generalization is to allow n (5 in the above exaniple) to take 
any value. Again, this opens up a whole new range of distributions. 

No attempt has been made to chronicle any significant number of the 
infinite number of possible distributions. The basic procedure is summarize! 
below. 

13 Obtain the characteristic equation corresponding to the feedback 

t 

shift register jof interest (assume its length is n ). 

23 IVrite dovn the sum S in terms of the first n values of 
• « 
eP (1 ^ j 5 ri3 • Use the characteristic equation to express values of 

for 3 <n in terms of values of iP for 1 ^ 5 £ n . 

33 Separate the 'terms in S into independent groups. 

43 Find the distribution of each independent group through the tabular 

method shown above. 

S3 Convolve the distributions of the independent groups to find the 
overall distribution of the sum. 
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3.3 Added Logic 

The approach used in section 3.2 suggests an alternative to adding 
additional stages. Since we can express any sum in terms of the first n 
values of we can obtain any sum by simply using logic C^odulo 2 adders) 
prior to analog summing. Assume, for example, that we wished to obtain the 

sum defined m equation (20) as: 

Sg = -J- + (D^ + D^) (24) 

V 

This sum is easily realized by the circuit shoivn in figure 6. 


Analog Adder 

output 






> 

. i 

i 

• 





4 

5 ■ 


1 

2 

3 



Figure 11. Synthesis of Sum Sg 


It is apparent that this basic approach can be used to synthesize any 
sum discussed in section 3.2. In a particular practical case, it would be 
necessary to decide whether it would be more desirable to use additional shift 
register delay stages or additional logic circuits. No general role seems 
to hold for all situations. 
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It is also apparent that the above scheme can be used with different 
weighting factors to produce a wide range of different distributions. 


4.0 Infinite Sums 

In this section we are concerned with a problem closely related to 
that of the first three sections. We seek the probability density function 
of the random variable: 

X = I a’' (25) 

k=l ^ 

v/here "a" is a constant, and is a random variable. The relation to 
our earlier work is evident when we write the sum as obtained in section 2.3 
as: 

s = i 2 '' 5, (26) 

k=l 

where is a pseudo-random variable which takes values 0 or 1 with nearly 
equal probability. 

Here we extend the problem to the infinite sum where ”a” can take any 
value and can be any random variable. The problem is solved with 
comparative ease for some cases and is not satisfactorily solved for other 
cases . ' We start with the easy case (a = binary with equal probability 

of either state) and consider later the more difficult cases. 

4.1 The Uniform Case 

We coiisider first the random variable: 
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X = I 

k=l ^ 




( 27 ) 


where Cj, is a binary random variable which takes the values c and d 
with equal probability. That is: 


PCCj, = c) == P(5j, ^ d) = i , for all k 


(28) 


We assume fiarther that the are independent. This problem has 

* 

been solved (for c =» 0 and . d = 1 ) by Blake and Thomas , Hence, we 

4 w- 

are working with the problem of the infinite sum of independent random 
variables of the form a . We will use a characteristic function 
approach. It is well-kno\m^^^^ that the characteristic function of a sum 
of independent random variables is the product of the individual characteristic 
functions. 


K(w) = n ({., (w) 
^ k=l ^ 


V 

- 4’j-(w) , the characteristic function of a , is: 


(29) 


(30) 


Applying (29) we obtain: 


^„(w) = n ^ ~ ^d-^ 

^ k=l 


k+1 

k=l 

w(d-c) 


^+ 1 ~ 


iw(c+d)/2 — «- 

= e-' ^ sin 2 

w(d^cj 


(31) 


(32) 
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The two infinite products of equation (31) are found as indicated in 
equation (32) , in the appendix. The characteristic function of equation (32] 
is that of a uniformly distributed random variable with range from c to d. 
That is: 


1 

d-c * 


c < X < d 


fxCx) = 



othen^ise 


(33) 


This is a most interesting result, namely that the infinite sum in 
equation (27) is uniformly distributed. This .problem has been considered, 
though not originally in a probabilitie framework, for a long time. It 
was shorn by Vieta (see Kac^^^^) that any real number t , 0 ^ t ^ 1 , 
can be expressed uniquely as: 





(34) 


where is either 0 or 1 for each j 
some detail by Kac. 


The problem is discussed in 


4.2 General Weights (a = 1/2) 

We turn now to the general problem of the distribution of: 

X ■= I a’' 5, C3S) 

k=l ^ ' 

where -a can be any number from 0 to « . This problem is not solved 
here in any closed form. Rather we suggest some of the results and point 
out problems which are still under consideration. 

We are tempted of course to simply apply the transformation approach ' 
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used in section 4.1. This leads us to a transform of X given by: 




n cos w(d-c)a^^^ 
k=l 


(36) 


It is seen that the first infinite product, involving exponentials, 
reduces to an easily expressed ’’phase" term. The second product, however, 
is not knom to have a simple solution. So the method of section 4.1 does 
not lead to the closed form solution for which we might hope. In fact, 
further consideration of the range of X _ suggests some very real problems. 

a) Case 1: 3. < ^ 

When a = Y ^ is a continuous random variable on the range [c,d]. 

However, if a < ^ it is not possible to obtain all values of X bet^-reen 

1 

the minimum and maximum. In analogy to the formula of Vieta (see equation 34) 
consider for following expression. . 

12 3 

t = e^a + e2a + e^a + • • • (37) 


If e. can be either 0 or 1 the range on t is -0 t 
For example, if a = ^ , we have: 


®1 ®2 ®3 


(38) 


and the range on t is [0,j]. However, there are many values of t which 
cannof be generated by any combination of 's . For example, none of the 
numbers in the open sub-range (•g-,^) can be obtained. In fact, the closed 
range [0,j] has an infinite number of open sub-ranges in which no number 
can be generated by equation (38) with any combination of e^’s . 
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Let us turn to a consideration of the distribution of F . This 
problem is discussed by Feller. As we shall see it turns out to be 
essentially meaningless to speak of the probability density function (PDF) , 
so we shall look at the cumulative distribution function (CDF) . The CDF 

NJ 

has an infinite number of flat regions separated by jumps which are 

infinitely small. The largest flat region is Cp^) which is centered at 

X = ^ and had a length of ^ . There is also a flat region centered 

about X = ^ with a length of ^ , and another of length ^ centered 
7 

at . The CDF for these regions only is plotted in figure 12. The CDF 
for these regions plus the next set of regions between these is shown in 
figure 13. As the number of terms in equation (38) taken into account is 
increased, the size of the jumps in the CDF, F(X), decreases. (See figure 14‘.) 
These jumps go to zero in the limit as all terms are considered. Hence, the 
CDF becomes continuous, and yet no PDF exists. Feller refers to this 
as a singular distribution. 

The example above is for a = ^ but the same type of distribution arises 
for all 0 < a < ^ . Work is presently underway to effect a more meaningful 
representation of the distribution for this case. 


Case 2: ^ a < 1 

We consider now the range of t in .equation (37) under the assumption 

1 a 

Y < a < 1 . The maximum and minimum values are 0 and , The range 

on t is continuous; that is, every number t between 0 and can 

be expressed as in equation (37) with appropriate ©j's . To see this -s^e 

consider the way in which we generate appropriate values of e^ ’s for a- 

given t . 

The procedure is to subtract from t the largest value of a^ which 
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0.0 1 1 , i ^ ( »— 

0123456789 
Figure 12, Partial CDF of X (3 open regions shoun) 



Figure 13 


Partial CDF of X (7 open regions show*) 
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is less than t . For this value of j ive let = 1 . For each smaller 

value of j we let = 0 . We now try to subtract from the' new 

difference t - . If we can do so without leaving a negative remainder, 

we let = 1 , i£ not then " 0 . We proceed in like manner with 

etc., generating new values of based on whether or not the 

i+k 

corresponding a*’ can be successfully subtracted from the new difference. 
We continue until the last difference is zero. This may be possible only 
in the limit as the number of terras approaches infinity. 

Suppose that at any point in the procedure the original number t 'or 
a new difference t - a^ - a^^^ ~ ••• falls, as it must, within the range: 


_n 


< t < a 


n+l 


(39) 


A necessary and sufficient condition that the procedure can he carried out 
with an eventual difference of zero (if only in the limit as the number of 
terms is unbounded) is that: 

, • n+2 

t - < I a3 . » (40) 

“ ]»n+2 ^ ® 


for every t and every n . Equation (40) simply says that after any 
a subtraction the terms left to subtract must add to a number at least 
as large as the new difference. The largest value of t in the range given 
by (59) is so the worst case in (40) is: 

n . n*l 

a - a iirrf 
vhich reduces to 


(1-a)^ < 


(41) 





Note that the equality in C41) is met for a = ^ , but the inequality 
fails for a < . This is in agreement with the argument of the preceding 


section. Ho^^ever, the inequality holds for 4 < a < 1 . Hence, any 
a 

1-a 


0 < t < can be realized by a proper selection of e- ’s as long as 


1 ' 

a 1 . Thus, we have established that the range on t is continuous 


foi 


< a < 1 . 


We now direct our attention to the random variable defined by equation (35) 

The range on X is the same as the range on t ; follo\d.ng the argument 
s. 

above, 0 ^X 1 » and X is continuous in this range. 

h’e consider next the distribution of X . For a = j , X was found 

t 

to be uniformly distributed. More precisely, the expression, 


X - y c, = 0 m 

H ^ 


can be satisfied only by a unique set of 's for a particular X . That 
is , there is one and only one combination of ‘ s in equation (35) for 
a given value of X in the range [0,1]. 

If ^ < a < 1 this situation no longer holds. As an example let 
a = 0.6. There is only one way to obtain X = 0 ; it is necessary that 

f 

= 0 for all j . Likewise, we obtain X - 1.5 only if ~ ^ 
for all j . However, we can obtain numbers in between in many i^ays. The 
number 0.96 is obtained from the set Cj = 1 » ^ 1 

j > 2 . It can also be obtained by letting I * ^2 ~ ^ 

(for j > 2) take on any appropriate values to add^ to 0.36 . That this 
is possible is demonstrated by the fact that: 



j=2 


( 43 ) 
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which is greater than 0.36 . 

Hence, we have sho^svi that for ^ ^ 3. < 1 the random variable X is 
continuous and is not uniformly distributed. Work is continuing on the 
determination of this distribution. 

We close this section mth the comment that we believe that as "a" 
approaches 1, the distribution of X will probably become Gaussian-lihe. 

Case 3: a = 1 

For this case consider equation (35) with finite range. 

N 

X = I (44) 

1^1 

This is simply the problem of N independent trials and the distribution 
of X is of course binomial of order N . As N increases without bound 
the distribution of X asyptotically approaches a Gaussian distribution, 
according to the D&^Ioivre -Laplace Theorem. 

Case 4: a = 2 

Again, we let the number of terras be finite 

X - Z 2^ C, (45) 

h-0 ^ 

As was sho^<jn in section 2.3, the resulting distribution of X is 
uniform. - . 


Case 5: l<a<<», a42 


Little work has been done on the mde range of "a" greater than 1. 
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There is, however, an intriguing symmetry or mirror effect of the type of 
distribution with respect to a *= 1 . Note again that for a = 1 the 
distribution is approximately Gaussian. For a = ^ and a = 2 , the 
distribution is uniform. There is a strong temptation to suggest an analogy 
between the ranges (0,-^) and (2j“) We are continuing to explore these 
relationships. Figure 15 summarizes the above 5 cases, including the 
speculations suggested for case 5. 
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6.0 Appendix: Infinite Products 

V+-I 

1) Simplify n 

lc=l 

» pwCc+d)/2^'^^ ^ gja(c+d) J -f ^ + . . .) 

k=l 

= ^5w(c^d)l I 
® ^ n=0 ^ 

= g3w(c+d)/2 (A-l) 

Since J (4)^ = 2 
n=0 ^ 



As the* trigonometric identity is continually applied to the last (sin 
Y term, that term approaches unity and the infinite product becomes: 




(A-4> 
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which suggests that* 


n 

h=l 


wCd-c) 
“St 


w^(d-c) 
sin 2 
w(d-c) 

2 



